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ABSTRACT 

The main document, of which this report is a summary, 
covers the assessment activities of the Minnesota Department of 
Education from June 1971 through December 1972. This summary report 
contains a chapter-by-chapter overview of the full report, with a 
concentration on the conclusions and recommendations drawn from the 
reading, mathematics, and attitude findings of the study. (JF) 



ERIC 



MINNESOTA EDUCATIONAL ASSESSIVIENT 
PILOT PHASE RESULTS 



y $ 06»»AftTMINT0F MiALTH. 
fOUCATJONIWftrARt 
NATIONAL kNiTITUTtOF 
fDUCATlON 

TMiS OOCUMENI MAS BEEN REPRO 
OUCCO EXACTLY AS R^CClVEO FROM 
1 HE PERSON OR ORGANIZATION OftjOlN 
IrtUOiT POINTS OF VIEW O'i OPINIONS 
STAieO 00 NO! NECESSARILY REPRC 
SCNTOf FICIAL NATIONAL iNSTlTUTC OF 
EDUCATION POSITION OR POk-iCY 



SUMMARY REPORT 



Prepared By: 

Dr. John W. Adams, Director 
State Educational Assessment 

Dr. Randall E. Johnson, Consultant 
State Educational Assessment 



Minnesota Department of Education 
Division of Planning and Development 
February, 1973 



SUMMARY RLPORT 



The Minnesota Educatlona) Assessment: Pilot Phase Results Report covers 
the assessment sctivittes of the Minnesota Department of Education from 
June 1971 through December 1972. Tills sunmi/jry report contains a chapter 
by chapter overview of the full report, v/ith a concentration on the con- 
clusions and recommendations. 

CHAPTER I . AN fNTRODUCTION 

The Governor's Advisory Committee on Education for the Seventies recommended 
in its Report and Reco mme ndations that "an educational assessment program 
for Minnesota should be initiated," At all levels of education the citizens' 
demand for '^accounlab i li ty" has been heard and many efforts to meet that 
demand' have been undertaken. 

Commissioner Howard 8. Casmey has concluded that these needs are of para- 
mount importance and has designated the development of 0 statewide assess- 
ment program as the top priority state department planning activity. The 
results of this assessment effort are expected to be presented to the 1973 
Legislature as the primary basis for support of an on-going state program. 

During fiscal year 1971, the Division of Planning ond Development: began 
planning the educational assessment program within the Department of Edu- 
cation, and solicited viewpoints and participation from a nu^nber of educa- 
t i onaKorgani zati ons in shaping the plan of attack. As a result of these 
discussions, the following specific objectives were assigned to the assess- 
ment program: 
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(1) To determine the level of performance of students In this state 
In the cognitive^ affective and psychomotor domains. 

(2) To Identify the variables which account for the variations In 
student performance. 

(3) To report the results of this Investigation to educational deci- 
sion-makers In the Executive and Legislative branches of state 
government, the State Board of Education, the Department of 
Education, local school adiTiInlstrators, local school boards and 
Interested citizens of the state providing a guide for the alloca- 
tion of school resources. 

(4) To longitudinally report the extent to whfch progress Is being 
made in Minnesota schools toward improving student performance 
within the State of Minnesota. 

The first year of the program was concerned primarily with the first ob- 
jective and secondarily with the third, it was assumed that until rep- 
resentative educational objectives and valid measurement techniques could 
be established, it would be neither feasible nor appropriate to pursue the 
second objective. It was further assumed that the results of the pilot 
year should not be considered complete enough to serve as a guide for 
resource allocation. Objective number four, of course, will be addressed 
over a period of time. 
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CHAPTER II. THE PLANNING PROCESS 



A two-phased approach to the planning, process Is described. The Dete r"' 
ml net Ion Phase included a review of assessment activities In other states 
an exploration of existing data sources, the formation of advisory groups 
and presentations of Departmental plans to interested groups. 

The Pi lot Phase focused on the developrnent of an objectives based model 
for assessing ach ieven^ent In reading and matheiiM^t i cs at the end of grades 
three and six. A procedure for identifying attitudincil measures followed 
a plan similar to that used for reading and nathematics. The Pilot Phase 
model outlined the foMowi ng steps: 

(1) Identification of relevant i nstruct iona ) objectives in the areas 
of reading and mathematics at the end of ^'^rades three and six, 
determined by key msth and reading educators ^">f the State. 

(2) Acceptance .is relevant instructional objectives by school 
personnel in the State. 

(3) Acceptance as desirable objectives by citizens of the State con- 
cerned wl ^h education. 

{k) Development of specifications for i ps t rumen tat I on which v/ould 
measure if and/or to what extent these objectives are being 
achi cved . 

(5) Identification of educational needs, defined as the discrepancy 
between desired and actual performance. 

The Development of ti\e Sampling Plan was guided by recommendations from' 
the Assessment Mvlf^ory Couricil. A five per -cnt rarjdom sample of third 
and sixth grade classrooms of the state was tf^e pUsn recor^mended by the 
Council and implemented by the Department. 



The Assignment of Tests to tfie students selected in the sample was based 
on a matrix sampling plan v-^hich resulted in any one student receiving only 
a fraction of the total number of tost Items. A calendar of events Is 
Included In this chapter. 

CHAPTER 111. DEVELOPMENT OF OBJECTIVES AND INSTRUMENTS 

This chapter provides a detailed description of the developmental steps 
leading to the construction of reading, mathemotlcs, and affective instruments. 

The Minnesota Reading Assessment Instrument Wcis based upon the fundamental 
objectives for reading instruction set out by the Statewide Committee for 
Reading Assessment. These objectives represented an independent, logical analysis 
of reading as a developmental process. Four general testing domains arose 
from the accepted objectives: 1) Word Attack, 2) Word and Sentence Com- 
prehension, 3) Comprehension in Longer Discourse, and ^) Reading Study 
Sl<ills. Within each testing domain o set of subtests was created to measure 
reading skills designated as critical to j child's progress in any well 
conceived elementary school reading program. 

The task of developing an instrument for assessing the ma t h ema 1 1 c a 1 c ompe - 
tencles of third, and sixth grade students in Minnesota included the develop- 
ment of objectives and the writing of test items. 

Objectives were written for both grade levels and included the areas of 
Numeration, Addition and Subtraction, Multiplication and Division, Graphs 
and Patterns, Geometry, Measurement, Relations and Functions, Probability, 
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Sets and Logic^ and Problem Solving, The topic of Fractions was applicable 
only for the sixth grade. 

Due to time constraints placed on itemv/rlting it was necessary to 
limit the number of areas for testing. Numeration, Operations (whole 
numbers, fractions, and decimals): Addition, Subtraction, Multiplication, 
and Division, and Geometry were selected as the areas to be covered. The 
judgment to cover these topics was based upon their somewhat representa- 
tive nature. Operations covered the mainstream of computational skills, 
white some concept development was picked up In Numeration and Geometry. 

It was originally thought that the Minnesota Attltudlnal Survey could be 
drawn primarily from existing affective Instruments. However, review of 
the attitudinal tests used in other states revealed that none of them were 
totally appropriate for the general areas of Interest in Minnesota and 
also that the validity and reliability of such measures were not much 
greater than that which could be achieved in Minnesota after a year of 
pilot testing. Therefore, development of a Minnesota Attitude Inventory 
began at an earlier time than the development of any of the other objective 
referenced tests finally used in the pilot assessment. 

The limited outline which served as the basis for the generation of the 
assessment instrument focused on the folloiving eight areas: 

(1) Attitudes Toward School and School/Achievement 

(2) School Atmosphere 

(3) Self Concept 

W Attitudes Toward Reading 



(5) Attitudes Toward Mathematics 

(6) Citizenship 

(7) Attitudes Toward Careers and the V/orld of Work 

(8) Attitudes Toward the Environment and Ecology 

CHAPTER IV. IMPLEMENTATION AND ANALYSIS 

This chapter outlines the procedures used in the admi n i strat Ion > data 
handling, analysis, and reporting or the results of the pilot phase assess- 
ment . 

As a necessary step In the preparation of teachers to administer the 
assessment instrument, four regional in-service meetings for the classroom 
teachers selected in the sample were held in Minneapolis, St. Paul, Mankato, 
and Srainerd. At the meetings, the student packoges v/ere distributed to 
the teachers and administration procedures were reviewed w(th them. 

In order to minimize the fatigue factor for students, the te achers admini stered 
the three packages as foflov/s: 

(1) Tlie first package at the beginning of school. 

(2) The second package at mtd-morning. 

(3) The third package in the afternoon. 

The statistical analysis phase provided data for interpretation from the 
mathematics, reading and attitudinal tests. The area of mathematics pro- 
vided data by item within form, within grade, as well as across the forms 



for common Items within c\ grade. For r»:ic3dinn, it provided data by item 
within form, within grades, as well as by item, across forms within grades 
for those common items. In the attitudinal survey, the analysis pro- 
vided the percentage distribution of the responses for each Item by 
form v;i th i n grade . 

CHAPTER V. RESULTS 

The complete results in mathematics and for the attitudinal survey are 
given in this chapter together with the objectives which were being 
measured. For the reading results, only sample Items are given for each 
of the subtests. 

Only about one-half of the mathematics objectives were tested. The reading 
subtests covered all but the ^'higher mental processes", an area which was 
suggested in the reading objectives, but which was found too difficult to 
measure within the time given for instrument development. The reading sub- 
tests include word attack skills, word and sentence comprehension, and 
reading study ski lis. 

The attitudinal survey is limited to school and school related variables. 
Several other attitudes might hdve been measured, but for reasons of being 
outside of normal school influences or of having tittle Informational value, 
other attitude scales were not included. 

Teacher reactions are summarized in this chapter, Each of tfie th( rd ^nd 
sixth grade teachers were asked to complete an evaluation of the reading, 
mathematics > and attitudinal instruments* These evaluations w.-:ro included 
In the package of completed tests which were returned during the week of 
May 15, 1972. 



A five point rating scale was used for the responses to five questions 
which covered the clarity of the direct ions > the appropriateness of content 
and language, and the adequacy of time alloted. 

The ratings were averaged across aM of the respondents. The ratings were 
all above the "average to good" point (3^5) on the scale. For the atti- 
tudinal survey and mathematics i nstruments , the ratings were all In the 
"good to excellent" interval {k to 5) • 

The average time taken for the administration of each of the instru- 
ments showed that the reading test took nearly one hour, the mathematics 
tests about '^O minutes, and the attftudinal survey about 25 minutes. 

In addition to the rated responses, teachers were also Invited to comment 
on any aspect of the administration or character of the instruments. These 
comments are summarized. 

CHAPTER Vt. CONCLUSIONS AND RECOMMENDATIONS 

The conclusions which follow from the reading, mathematics, and attitude 
results presented In the previous chapter attempt to highlight only a few 
of the many conclusions which could be drawn. Each reader who wishes to 
examine the results in detail can probably find several more contrasts and 
points for discussion. The reader should be cautioned not to over generaliz 
f rem the res u 1 1 s and t he reby a r r i ve a t cone 1 us i ons wh i ch might be p r ema t u re . 
Since th<jre were no subgroiiping of students, no corrparlsons between groups 
are posslbi.^. The only observations possible are within the areas tested 
and in some cases between third and sixth graders* 



READING 

The reading results include the percentages of students responding correctly 
to each Item. Then mean percentages were computed by averaging over all the 
Items In a particular sub-test. The present Interpretation Is based on 
these data. It must be regarded as prellnlnary. A more expanded Inter- 
pretation must await further analysis of the data. 

Recognizing these reservations, the data present a reasonable picture of 
reading instruction in Minnesota. While the picture does not Illustrate 
educational perfection, It Is by no means gloomy. The following generali- 
zations seem warranted: 

(1) By the end of third grade, children are nearing mastery of most 
of the word attack skills tested in the assessment. End of the 
year sixth graders have improved over end of the year third graders 
on every skill, with the greatest improvement coming in the most 
difficult word attack skills. 

(2) There is considerable growth between grades three and six on comp- 
rehension skills, perhaps reflecting a shift of emphasis from word 
attack to comprehension in intermediate grade Instructional programs 

(3) In terms of the study skills tested, neither third nor sixth 
graders demonstrated the degree of mastery achieved for word attack 
ski 1 Is , perhaps reflect fng the amount of emphasis given to these 

^^^^^ ; s In the elementary curriculum. There is definitely a 

■ : ■ developmental trend; that is, sixth graders out-performed third 
^. >■■'■■; graders on all common tests. .■■>■•;■, 
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Regardless of the particular Intermediate grade program, teachers expose 
children to Increasingly complex written discourse and they require children 
to demonstrate that they have understood what they read. These dual emphases 
result In maximal growth for those tasks requiring attention to longer 
units of discourse. As the units of discourse decrease In length, the 
growth from grade three to grade six decreases. 

T^^^ study skills sub-'tests were developed to assess two basic types of 
study skills: (I) skills that directly follow from mastery of basic 
word attack skills and (2) skills that reflect a basic competency In using 
or finding textual or reference material. 

It Is clear that where common tests were given at both grades, progress 

in mastery of Type 1 Study Skills occurs between the end of grade three and six. 

Also, the order of the difficulty of vhe' commonly tested skills remains 

the same but by grade six there is considerable improvement In the most 

difficult sub-test , Guide Words. 



MATHEMATICS 

The conclusions from the mathematics testing are based largely on the 
results summarized in the table below. Since only about one-half of the 
objectives written for mathematics were tested, these conclusions should 
not be viewed as a complete measure of the competence of the third and 
sixth graders tested. 



DISTRIBUTION OF ITEMS BY PERCENT OF STUDENTS ANSWERING ITEM CORRECTLY 



Grade 3 

Number of Items by Percent Correct 
Total Number Less 

Content Area of Items 100-90 90-80 80-70 70-60 60"5O Than 50 

Numeration ~ Is 2 5~ V 3 H 0 " 



Addition and 

Subtraction 25 7 10 6 1 1 0 
Multiplication 

and Division 5 0 0 5 0 0 0 

Geometry 6 _r 2 1 0 I 1 

TOTALS 5I 10 17 TS 5 3 T 



Grade 6 

Numeration 

(With Fractions) 20 2 3 8 3 2 2 

Addition and „ 
Subtraction 9 3 I 2 2 0 1 

Hultlpl Icatlon 

and Division 10 0 1 3 3 2 1 

Geometry 5 2 0 1 0 0 2 
Fractions 

(Operations) 10 II 3 2 2 1 

TOTALS 5V 8 6 IT 10 6 7 



THIRD GRADE 

Numeration; 

The results In the area of numeration showed a wide spread of competency. 

The more difficult Items required a fami 1 larl ty wi th fractional symbols 

and finding a missing number I n a pattern of three-dlgl t 6ven and odd numbers. 



The Items which the students found easiest were related to the counting of 
objects, patterns of one and two digit numbers, and place-value Identification 

Addition and Subtraction s 

The performance of third graders on Items In the area of addition and 
subtraction of whole numbers was highest among all of the four content 
areas tested. 

: , . , • ' ■ .' A • ■ " 

Hult I pi I cation and Dlytslon : 

The five Items relating to the multiplication and division of whole numbers 
were answered correctly by between seventy and eighty percent of the students 
tested. By the end of the thl rd grade, students have spent much more time 
with addition and subtraction concepts than with muttlpl Icatlon and division. 

G eometry i " 

The pattern of results In the geometry area were similar to the area of 
numeration, where again the spread of competency was apparent. 

xth Gr^d^;^v^';::::;;;^ 

The results of the testing of sixth graders revealed a greater spread In 
performance than the results from the thl rd grade. As children progress ^ 
through the grades there Is an ever widening range of mathematical ski I Is 
and concepts to be acquired. Therefore, within the category of ^'numerat ion" 
the students see a variety of Items for which they have had varying degrees 
of exposure and practice. 



Numeration: 

Twelve of the 20 Items (n the numcrntfon oroa v/ere rinswered correctly by 
seventy percent or more of the students tested. However, four I terns were 
answered correctly by sixty percent or less of the students, Three of 
these four Items required operations with fractions nnd the ordering of 
f ract Ions . 

Addition and Subtraction ; 

The computational skill of sixth ciraders was evident In addition and sub- 
traction items where four of nine items were answered correctly by more 
than 80 percent of the students tested. The student performance declined 
somewhat in Items calling for the recognition of the concept of regrouping 
(associativity) and to identify addition and subtraction on the number line. 

Multiplication and Division: 

Only one of the ten I terns in the n^ultlpl ication and division area was 
answered correctly by more than 80 percent of the slxtli graders'. This 
Item required the oiul tipl I cat ion of a four digit number by a single digit, 
Items causing the greatest difficulty required the recognition of the 
correct regrouping of three factors (associativity). Only about fifty 
percent of the students answered these items correctly* 

Geometry ; 

An item calling for the recognl t ion of common geometric fl^^^os and another 
I tern requJ rfng the Identt flcation of a figure of largest area were answered 
correctly by over 90 percent of the students. In contrast , an i tern re-^^^^^^M^^^^^^^^^^^ 
qui ring th<s fdent 1 f icat Ion of elements related to a ct rcle was answered 
correctly by less than f I fty percent of the r> tudenf? * 



Fractions ; 

The ten I terns requiring computc^t. i on with fraction'^, produced a wide range 
of results. FWe of the ten responses foil botwoon sixty and eighty 
percent correct. The two items above eighty percent required the addition 
and subtraction of decimal fractions, whore tl\e placement of the decimal 
point was not required. Three items answered correctly by sixty percent 
or less of the sixth graders required ^,he addition and subtraction of 
common and mixed fractions. 

ATTITUDIHAL 

The content of each of the attltudlnal areas is described in Chapter V, 
Section D of the complete report. The conclusions followfng each of the 
eight headings below arc by no means the only observations which could be 
made from the results in the previous chapter. These conclu<;lons are stated 
In an attempt to lead and encourage the readers to explore the full range 
of the results and seek out information which is potentially useful. 

Attitudes Toward School and School Achievement : 

About three-fourths of the third and sixth graders surveyed expressed a 
liking for school. However, about three-fourths of the sixth graders 
thought school was "boring'^ at least some of the time. 

Marks (grades) wore ranked as ^'important" to ^'very important" by almost 
ninety percent of the students tested. Competition for marks or "doing 
better" than others in the class was held as "Important" to "very important*' 
for only about forty percent of the students* 
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School Atmosphere ; 

About twenty percent of the third graders viewed the principal's main 

Job as one of punishing students, while less than ten percent of the 

sixth graders held this viewpoint. However, when asked about 'Miking the principal 

to visit their classroom" over fifty percent of the third graders agreed, 

while only fifteen percent of the 6th graders agreed* 

The view of the teacher as a "listener" at least some of the time was 
held by nearly ninety percent of the students, while over one-third of 
the students felt the teacher "talked too much". Over seventy-five per- 
cent of the students felt they had to be "too quiet in class". 

Almost fifty percent of the students "didn't know'^ if other people In 
school cared about tliem. 

Self-Concept ; 

Both third and sixth graders are generally positive In viewing themselves* 
but about one-half of the population surveyed "did not know" if they were 
easily liked or made more mistakes than others. 

When asked about thei r ability to "learn", about one-thl rd of the stu- 
dents expressed some difficulty. Approximately eight-eight percent expressed 
a wl I I Ingness to take some r i sks wi th respect to "try Ing new things"* 

Attitudes Toward Reading : 

Over eighty percent of the students surveyed expressed a willingness to 
read at least "sometimes'* and over fifty percent "most or all of the time". 

When ^sked to chooso whatever they wanted to do In school , the pre* 
ference for reading was stated by about sixty^ pef'cont of the students at 

Q : ■ ■ ■ . ■ ■ 

uc 
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least ^'sometimes*'. About the same proportion of students ranked reading 
"as at least as important as any other subject". About fifteen percent of 
those surveyed ranked reading as "more Important than any other subject". 

When asked If they had a hard time remembering what they had read, one-half 
of the students expressed having this problem at least "sometimes". 

Attitudes Toward Mathematics ; 

More than one-half of the students "liked" or thought of arithmetic as "fun". 
Also, about one-fourth of the students viewed arithmetic as their **worst" 
subject. 

About ninety percent of the sixth graders and about seventy percent of 

the third graders ranked arithmetic as "at least as Important" as any other 

subject. 

Attitudes Toward CI tizenship ; ^ 
Eighty percent or more of the students surveyed agreed that "rules" were 
necessary and to be obeyed by them. However, fewer students, about 
seventy percent, agreed that grown-ups need "rules". 

Voting rights, and the right to go to school for other races and rel Iglons 
was agreed to by about ninety percent of the students , while about one- 
half of the students were undecided about being "represented" In the govern- 
ment by a person of another race or religion. 

When asked about the exercising of constitutional rights such as the right^^^^^^^^^ r 
to "plan demonstrations and marches to change things they don Vt 1 ike about^ ^^^^c 
the government" the opinions were evenly divided between "agreelng'Vand 



-17- 



'^disagreeing*'. About one-fourth of the students surveyed said, "I don't 
know"» 

Career Awareness; 

The Importance of being Involved In the *\vorld of work** Is a very commonly 
held value or attitude of both third and sixth grade students. Over eighty 
percent of the students surveyed agreed with statements which linked 
"hard work" with "feeling good", "being responsible", "getting what one 
wants from life" and "success'^ 

Envl ronmental Att I tudes ; 

More than two-thirds of the students agreed that time should be spent 
learning about environmental problems. When surveyed about taking positive 
steps to Improve the environment such as "picking up trash" fewer students 
were willing to participate. The change from third to sixth grade was 
noticeable with the sixth graders showing the greater concern. 

Recommendations 

Before listing a set of recommendations which have emerged as a result of 
the pilot phase of the assessment actlvi ties , the following observations 
can be made based on this report: 

1) CI tlzens , school professionals , legislators, university and col lege 
personnel V ^nd State Department of Educotlon staff gave thef r support 
when asked to become Involved In the assessment program. 

2) Minnesota school administrators, tec^chors , and s tudonts. cooperated 
ful ly when asked to be part of the pilot phase testing. 
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The following specific recommendations result from a review of the pilot 
phase acti vl ties : 

1) Guidelines need to be developed for specific subgroups of 
students relating to school > community, and family characteristics, 

2) The matrix sampling approach which provides information on groups 
of students as opposed to Individuals appears to be a viable 
solution to an efficient and economical method of collection of 
assessment data. 

3) Full Implementation of the assessment program will necessitate 
the development of student assessment measures In areas which have 
been found to be difficult to measure. Therefore, the Department 
needs to allocate additional funds to develop these measures. 

h) A review of the teacher reactions to the assessment shows that even 
though complete Instructions were given> there appears to. be enough 
variation In the testing time allotments so as to question the 
standardization of administration. Therefore, It Is recommended 
that the Department explore the poss 1 bi 1 1 ty. of using a pre-recorded 
tape to pace students through the assessment exercises in a uniform 
procedure. 

5) A fuller Implementation of the assessment program wl 1 1 require a 

substantial amount of local district personnel time* The Department 
: therefore should explore the feasibility of utilising trained 
assessment administrators paid by the Department to conduct the 
necessary field work and thereby rel ieving the local district of 
this burden. 
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An assessment program requires a fully Integrated systematic 
approach Involving a wide range of expertise In the planning, 
Implementation, and dissemination of an ongoing assessment program. 

As additional content areas are developed for assessment purposes 
It Is likely that many of the materials and technology developed 
by the National Assessment of Educational Progress (NAEP) will be 
useful to tho Department. This procedure would allow for the most 
cost effective utilization of assessment resources In Minnesota. 



